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Executive Summary 

This study is an evaluation of Extended Learning Opportunities — Summer Adventures in Learning (ELO 
SAIL), which is a Montgomery County Public Schools (MCPS) summer program in Title I elementary 
schools. ELO SAIL has two major goals: 1) to prevent the achievement loss that students may 
experience in summer and 2) to prepare students for the next grade level. The program is offered in all 
24 Title I schools during July; its target includes students who will enter kindergarten, Grade 1, or Grade 
2 in the fall after the summer session. The program features a four-hour instructional day of reading, 
language arts, and mathematics. 

The evaluation focuses on the impact of ELO SAIL on student academic achievement. This report 
addresses the following questions; a subsequent report will analyze the impact of the 2015 ELO SAIL 
program in greater detail. 

1. What were the demographic characteristics of students in Kindergarten-Grade 2 who attended ELO 
SAIL in 2012, 2013, 2014, and 2015? 

2. How did the students who participated in ELO SAIL in 2012, 2013, 2014, and 2015 perform in the 
fall, compared with students in Title I schools who did not participate? Did the academic impact of 
the program vary by student subgroups? 

3. How did the students who participated in ELO SAIL in 2012, 2013, and 2014 perform in reading 
and mathematics at the end of the school year, compared with their nonparticipating peers in Title I 
schools? Did the academic impact of the program vary by student subgroups? 

Summary of Methodology 

The study populations were all students who participated in ELO SAIL from 2012 to 2015. The 
comparison groups were students who did not participate in ELO SAIL and were enrolled in Grades K- 
2 in Title I schools during 2012-2013, 2013-2014, and 2014-2015. Rasch unit (RIT) scores in 
mathematics from Measures of Academic Progress-Primary Grades (MAP-P) were used as outcome 
measures for mathematics. Reading levels from the Assessment Program in Primary Reading (AP-PR) 
were used as outcome measures for reading. For questions 2 and 3, bivariate tests of significance were 
used to examine differences by grade level and content area between students who participated in ELO 
SAIL and those who did not for all students and for seven student subgroups: Asian, Black or African 
American, Hispanic/Latino, White, and students who received the following services: English for 
Speakers of Other Languages (ESOL), Free and Reduced-price Meal System (FARMS), and special 
education. 

Summary of Findings 

Question 1. Across four summers (2012, 2013, 2014, and 2015), more than 14,000 students in 
Grades K-2 attended ELO SAIL. Out of this total, about two thirds were Hispanic/Latino and one 
quarter were Black or African American. About one half of the students received ESOL, five out of six 
received FARMS, and one out of ten received special education services. The percentage for each 
subgroup was similar every year. 
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Question 2: Kindergarten. Among all kindergarten students, the only significant difference in 
fall AP-PR reading results was in one of the four years, when more ELO SAIL attendees were at a pre- 
reading level (i.e., have mastered foundational reading skills) compared to their peers who did not attend. 
The impact on fall reading results did not vary across kindergarten subgroups. In six of the seven 
subgroups (all except for White students), there were statistically significant differences for at least one 
year in favor of stronger reading performance by ELO SAIL attendees. 

Among all kindergarteners, the mean RIT score in mathematics from the fall MAP-P test was 
significantly lower for ELO SAIL participants than for nonparticipants in two of the three years 
examined. However, among subgroups of kindergarteners, the math performance in the fall of students 
who attended ELO SAIL did not differ from non-attendees, with very few exceptions. 

Question 2: Grades 1 and 2. The analyses for Grade 1 and Grade 2 students concerned changes 
in performance in the fall after each ELO session, compared to performance in the spring prior to that 
ELO session; four years were examined. The reading measures concerned whether students increased or 
stayed at the same AP-PR book level. Among all first graders, there was a significant difference in 
reading for only one year; ELO SAIL participants experienced less summer loss in reading compared 
with their peers who did not attend. Among all second graders, there were no differences between ELO 
SAIL attendees and non-attendees in summer reading loss for any year. There was little variation in 
summer reading loss by subgroups of first or second graders; ELO SAIL attendees did not differ from 
non-attendees, with very few exceptions. 

The mathematics measures for Grades 1 and 2 concerned gains in RIT scores in mathematics from MAP- 
P. For all Grade 1 students, the mean gains of ELO SAIL attendees were significantly higher than those 
of non-attendees in two of the four years. There was little variation in impact across subgroups of first 
graders. Differences in math gains were statistically significant in favor of ELO SAIL for at least two 
years among five of the seven subgroups: Black or African American, Hispanic/Latino, ESOL recipients, 
FARMS recipients, and special education recipients. 

Among all second graders, the mean gains in math scores were significantly higher for ELO SAIL 
participants than nonparticipants in three years. There was some variation by subgroup. Differences 
between attendees and non-attendees were statistically significant, in favor of ELO SAIL, in three years 
for two subgroups, ESOL recipients and FARMS recipients, but in only one or two years for three 
subgroups: Black or African American, Hispanic/Latino, and White students. 

Question 3: Reading. Reading performance at the end of the year was analyzed for three years; 
the only statistically significant differences for all students involved kindergarteners. More 
kindergarteners who attended ELO SAIL met the end-of-the-year AP-PR reading benchmark than non- 
attendees in two years. There was little variation by subgroup. There were significant differences for 
two years in favor of ELO SAIL for five of the seven subgroups: Black or African American, 
Hispanic/Latino, ESOL recipients, FARMS recipients, and special education recipients. 

First and second graders who participated in ELO SAIL performed as well in meeting the end-of-the- 
year reading benchmark as their peers in Title I schools who did not attend. Likewise, for most 
subgroups of first graders and second graders, end-of-year reading performance did not differ between 
attendees and non-attendees in any year. An exception was statistically significant differences in favor 
of ELO SAIL attendees in one year for each of the three services subgroups. 
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Question 3: Mathematics. In analyses of all students, ELO SAIL participants had lower RIT 
scores on the MAP-P mathematics test at the end of the following year, on average, than their 
nonparticipating peers for each grade level in each of the three years examined. These differences in 
mathematics were statistically significant in favor of non-attendees in each year for Grades 1 and 2 but 
in only one year for kindergarten students. 

In contrast, among most subgroups in most years, ELO SAIL attendees and non-attendees did not differ 
on end-of-the-year mathematics performance. However, there were a few statistically significant 
differences in favor of ELO SAIL attendees and a few statistically significant differences in favor of 
non-attendees. 

Conclusion 

In conclusion, the positive impact of ELO SAIL was stronger on academic performance in the fall than 
on performance at the end of the year, based on statistically significant differences between attendees 
and non-attendees. Consistent with previous evaluations of ELO SAIL, the benefits varied by content 
area, with more positive findings in mathematics than in reading, and varied by grade level, such that 
positive findings in reading mainly were for kindergarteners, while positive findings in mathematics 
were limited mainly to first and second graders. However, in this study, unlike previous ones, a benefit 
for an entire grade level usually was evident for the majority of subgroups. 
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Impact of the Extended Learning Opportunities Summer 
Adventures in Learning Program (ELO SAIL) on Student 
Academic Performance: Part 1, Results From Fall 2012 to Fall 2015 

The Office of Shared Accountability (OSA) is conducting an evaluation of Extended Learning 
Opportunities — Summer Adventures in Learning (ELO SAIL) in Montgomery County Public 
Schools (MCPS) at the request of the Office of the Chief Academic Officer. The mission of MCPS 
is to ensure that every student will have the academic, creative problem solving, and social 
emotional skills necessary for success in college and career, as reflected in the district’s Strategic 
Planning Framework (MCPS, 2013). Achieving this mission requires eliminating achievement 
gaps among student groups. Extended school year and extended school time programs in MCPS 
provide students in Title I schools additional learning opportunities to further develop academic 
background knowledge. This document is one of two that examine the impact of ELO SAIL on 
student academic achievement. This report includes ELO SAIL sessions from summer 2012 
through summer 2015 and analyzes data from fall 2012 through fall 2015; the next report will 
focus on the summer 2015 ELO SAIL session and analyze data available after fall 2015. 

Background 


Program Description 

Overview. ELO SAIL is a free summer program for students in Grades K-2 in MCPS 
Title I elementary schools. The major goals of ELO SAIL are first, to prevent the achievement 
loss that students may experience in summer, also known as summer loss, and second, to prepare 
students for the next grade level. ELO SAIL was piloted at 17 Title I schools in 2002 and continued 
to operate in all 24 Title I schools in summer of 2015 as a part of the district’s strategy to focus 
resources and supports for students in these schools (MCPS, 2015). 

Target population. The target population for the ELO SAIL program includes students 
who will enter kindergarten, Grade 1 or, Grade 2 in the fall after the ELO SAIL session. All 
students in Title I schools are eligible to attend. Participation is voluntary in nature. However, 
student attendance is monitored closely. 

Program purpose. The ELO SAIL program is an equity strategy that seeks to serve the 
following specific purposes: 

• Meet the academic needs of each participant by providing opportunities to review grade- 
level concepts 

• Accelerate learning by previewing concepts and skills to be taught in the grades students 
will enter in fall 

• Strengthen basic skills that are preconditions of later learning 

• Mitigate the achievement loss that students may experience in summer 

• Provide continuing English language instruction for speakers of other languages 
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Structure and funding. In 2015, the ELO SAIL program was conducted for 19 days in July 
and featured a four-hour instructional day of reading, language arts, and mathematics. Bus 
transportation, breakfast, and lunch were provided free of charge. ELO SAIL is a core program 
of the federal Title I programs and is supported mainly with federal Title I funding. 

Staffing. All teachers must be highly qualified in order to teach in the ELO SAIL program. A 
“highly qualified teacher” is someone who has a [minimum of a] bachelor’s degree from an 
accredited institution, demonstrates competence in the subject area in which he/she will teach, and 
is certified to teach in the state of Maryland (Maryland State Department of Education, 2015). In 
summer 2015, classroom teacher to student ratio was 1:18 for all grades. Each school also had 
one teacher for English for Speakers of Other Languages (ESOL), one special education resource 
teacher, one paraeducator for every 75 enrolled students, one summer attendance secretary, one 
cafeteria support, and one clerical support for registration. The Office of Community Engagement 
and Partnership in MCPS recruited 230 volunteers to assist staff for ELO SAIL in summer 2015. 
The number of volunteers assigned depended on ELO SAIL enrollment in the school, ranging from 
4 to 15 volunteers per school. 

Major program changes since 2009. In 2009, ELO SAIL targeted students entering 
kindergarten through Grade 5 in 22 Title I schools, used an earlier MCPS curriculum, and had 
staff-student ratios between 1:17 and 1:19 in Grades K-2. Due to fiscal constraints, the program 
was limited to Grades K-2 in summer 2012. Although stimulus funds permitted ELO SAIL to 
again serve Grades K-5 in summer 2013, MCPS responded to the threat of economic instability 
by committing to fund the program for the early learners K-2 from summer 2014 onward. In 
summer 2015, ELO SAIL served Grades K-2 in all 24 Title I schools, provided instruction with 
MCPS Curriculum 2.0, and had a staff-student ratio of 1:18 in Grades K-2. 

Expected student outcomes. The short-term goal of the ELO SAIL program is to provide a 
stimulating academic summer opportunity for students in Title I schools. Further, the expectation 
is that students who attend the ELO SAIL program will maintain or improve their skills in reading 
and mathematics and be ready to succeed at the next grade level. In the long-run, the expectation 
is that ELO SAIL attendees will continue to make expected academic progress and to meet or 
exceed grade-level benchmarks in reading and mathematics. 


Previous Evaluations of ELO SAIL 

There have been MCPS evaluations of ELO SAIL for three previous years of the program: 2002, 
2003, and 2007. 

ELO SAIL began in summer 2002; the program was for 20 days and provided a preview of reading 
and math concepts to students in kindergarten through Grade 3 at 18 schools. The first year of 
ELO SAIL was evaluated for its impact on student academic outcomes and whether the impact 
differed across student subgroups of race/ethnicity and services received (Sunmonu, Larson, Horn, 
Cooper-Martin, & Nielsen, 2002). Students completed pretests in mathematics and reading at the 
beginning of the ELO program and posttests in September after the program. Sunmonu et al. found 
modest but statistically significant benefits in mathematics for 1 st , 2 nd , and 3 rd graders who attended 
all four weeks of the program and modest benefits in reading for students in Grades 1 and 2. The 
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ELO SAIL program benefits were similar across all ethnic groups and also apparent for students 
who received Free and Reduced-price Meals System (FARMS) and ESOL services. However, in 
Grade 2, the ELO SAIL benefits in mathematics were limited to students with the lowest academic 
needs, while the benefits in reading were limited to students with the highest academic needs. 

Sunmonu, Curry-Corcoran, and Mordica (2004) evaluated both implementation and outcomes for 
the second year of ELO SAIL in 2003, when the program served students in kindergarten through 
Grade 4 in 18 Title I schools and ran for 20 days. These evaluators found that student recruitment 
efforts and attendance strategies were largely successful; credential for the teachers at the summer 
program were equivalent to teachers in Title I schools. Program administrators and teachers, along 
with parents of attendees, were satisfied with ELO SAIL. To evaluate outcomes, the authors 
calculated the change in each student’s test scores from the spring prior to ELO SAIL to test scores 
from the fall after the program for students in Grades 1 — 4; they used fall scores only for 
kindergarteners. Students who attended the program for at least 16 days had higher spring to fall 
improvements, compared to their peers who attended five days or fewer, including those who did 
not attend at all. Although the only statistically and practically significant difference overall was 
for Grade 4 students in mathematics, the benefit did vary by student subgroup. The authors found 
significant benefits, in selected grades and academic areas, for ESOL recipients, FARMS 
recipients, and also for Asian, Black or African American, and Hispanic/Latino students. 

Wang (2009) evaluated the 2007 ELO SAIL program for implementation and impact on student 
academic achievement using multilevel data collection methods. She found significantly higher 
reading performance in fall for Grade 1 students who attended all four weeks of ELO SAIL (fully 
participated). Grade 4 students who attended all four weeks of ELO SAIL significantly 
outperformed their nonparticipating peers in mathematics. While not all racial/ethnic groups in 
Grades 1 and 4 showed these significant differences, students who were recipients of ESOL or 
FARMS services benefited more from ELO SAIL than non-recipients for both first and fourth 
graders. 

In summary, the previous MCPS studies found a small to modest positive impact of ELO SAIL on 
student fall academic performance in reading and mathematics for students with high attendance 
at the summer program. However, the benefits varied by grade level, content area, and student 
subgroup. 
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Scope of the Evaluation 

This evaluation examines the impact of ELO SAIL on student academic achievement as posed by 
questions that were developed in collaboration with the program administrators in the Office of 
Title I Programs. This report presents findings on the following evaluation questions; a second 
report will analyze the impact of the 2015 ELO SAIL program in greater detail. 

Evaluation Questions 

1. What were the demographic characteristics of students in Kindergarten-Grade 2 who attended 
ELO SAIL in 2012, 2013, 2014, and 2015? 

2. How did the students who participated in ELO SAIL in 2012, 2013, 2014, and 2015 perform 
in the fall, compared with students in Title I schools who did not participate? 

a. Did kindergarten students who attended an ELO SAIL program perform better in reading 
and mathematics in the following fall, compared with their peers in Title I schools who did 
not attend? 

b. Did Grade 1 and Grade 2 students who attended an ELO SAIL program experience less 
summer loss in reading and mathematics, compared with their peers in Title I schools who 
did not attend? 

c. Did the academic impact of the ELO SAIL program vary by student subgroups, such as 
race/ethnicity and services received? 

3. How did the Title I students who participated in ELO SAIL in 2012, 2013, and 2014 perform 
in reading and mathematics at the end of the school year, compared with their nonparticipating 
peers in Title I schools? 

a. Did the program participants in kindergarten, Grade 1, and Grade 2 perform better in reading 
and mathematics in the following spring, compared with their peers in Title I schools who 
did not attend? 

b. Did the academic impact of the ELO SAIL program vary by student subgroups, such as 
race/ethnicity and services received? 
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Methodology 


Study Populations 

The study populations were composed of all students who participated in ELO SAIL from 2012 to 
2015. To answer evaluation questions 2 and 3, the comparison groups were students who did not 
participate in ELO SAIL and were enrolled in Grades K-2 in Title I schools during the following 
school years: 2012-2013, 2013-2014, and 2014-2015. Program participants from summer 2012 
to 2015 were chosen because these are the most recent cohorts with comparable and more complete 
assessment data. 

Measures 

Measures for this study included academic performance and student characteristics. 

Mathematics. Measures of Academic Progress-Primary Grades (MAP-P) is an integrated 
collection of computerized assessments designed for students in kindergarten through second 
grade (Northwest Education Association, 2008 and 2011). It includes multiple-choice items and 
a variety of other item types. Task instructions are provided through audio headphones to capitalize 
on non-readers’ or early readers’ auditory comprehension more than reading comprehension. The 
MAP-P are designed to provide educators with instructional information about what students are 
ready to learn. Rasch unit (RIT) scores for mathematics from MAP-P were used as outcome 
measures. 

To answer evaluation question 2 for mathematics performance in the fall, the measures varied by 
grade level. For kindergarteners, the measure was the RIT score from the MAP-P in mathematics 
assessment completed in the fall after the ELO SAIL session. For Grades 1 and 2, the analyses 
examined differences between the RIT score from the fall assessment following ELO SAIL and 
the RIT score from the spring prior to ELO SAIL. The measure to answer evaluation question 3 
was the RIT score from the spring (i.e., the end of the year) following each ELO SAIL session. 

Reading. Data from the Assessment Program in Primary Reading (AP-PR) were used as 
reading measures. The AP-PR is a research-based and locally developed assessment used to 
measure important concepts and skills in the MCPS reading curriculum. AP-PR results provide 
formative and summative data. The Text Reading and Comprehension (TRC) subtest of the AP- 
PR is administered in the fall, winter, and spring each year to students from kindergarten to 
Grade 2. Teachers and staff can use the data to monitor students’ reading accuracy, oral retell, and 
oral comprehension and to guide instructional decisions. Students are expected to reach grade- 
specific benchmarks in spring (i.e.. Level 4 for K, Level 16 for Grade 1, and Level M for 
Grade 2). (MAP-P includes scores for reading as well as mathematics. However, most 
Grade K-2 students did not complete the reading portion of MAP-P from 2012 to 2015, and so 
RIT scores for reading were not available.) 

To answer evaluation question 2 for reading performance in the fall, the measures varied by grade 
level. For kindergarten, reading performance in the fall, after the summer of each ELO session, 
was analyzed. For the years of this study, relatively few kindergarten students (about 15%) had a 
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book level from the AP-PR in fall. Therefore, instead of analyzing each student’s book level, the 
analyses focused on whether students were reading (i.e., had a book level) or not. More 
information on reading performance was available for AP-PR in 2015-2016 for students who were 
not reading. There was information on whether students had passed print concepts (foundational, 
pre-reading behaviors), had attempted the print concepts test and failed it, or were not ready for 
print concepts. There was also information on whether they had passed reading behaviors, which 
is part two of print concepts. Therefore, 2015-2016 analyses of fall reading performance for 
kindergarten students used three categories: 

• Non-reading, for students who were not ready for print concepts or attempted but did not 
pass it 

• Pre-reading, for students who passed print concepts or reading behaviors 

• Reading for students who had any book level 

To answer evaluation question 2 about reading performance in the fall for students in Grades 1 
and 2, each student’s book level in fall (after ELO SAIL) was compared to his or her book level 
in the previous spring (prior to ELO SAIL). The analyses focused on summer loss by examining 
the percentage of students whose fall book level increased or stayed the same compared to their 
book level the prior spring, meaning they had no summer loss. Analyses for 1 st and 2 nd graders 
included only students with a book level from both spring and fall. 

To answer evaluation question 3 for reading performance at the end of the academic year, 
percentages of students meeting grade-specific benchmarks (noted above) in spring were used. 

Student characteristics. Demographic information on students included grade level, 
gender, race/ethnicity, and receipt of the following services: ESOL, FARMS, and special 
education. 


Data Collection 

Student-level data on participation in ELO SAIL, characteristics, and outcomes were downloaded 
from appropriate MCPS databases. 


Analytical Procedures 

Descriptive analyses addressed all evaluation questions. For evaluation questions 2 and 3, 
bivariate tests of significance also were used to examine differences between students who 
participated in ELO SAIL and those who did not. Specifically x 2 tests were used for the reading 
measures because they were categorical (i.e., students did or did not have summer loss, students 
did or did not meet the end-of-year reading benchmark). For the mathematics measures, t-tests 
were used because the RIT scores were continuous (i.e., had a wide range of values) and were a 
ratio scale (i.e., start at zero and had the same distance between values). 
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Strengths and Limitations of the Methodology 

One strength of this study is that ELO SAIL attendees were compared to other students from the 
Title I schools that offer ELO SAIL. Thus, the comparison group had similar school experiences. 
Further, the authors calculated the academic performance of multiple, successive cohorts of 
students (for four years for fall performance and for three years for end-of-year performance) to 
estimate the yearly impact of ELO SAIL on student academic achievement. The inclusion of 
multiple years of results for each question leads to increased confidence in the study’s results and 
conclusions. 

There are three caveats when interpreting the results. One, the tests of significance did not control 
for any differences between attendees and non-attendees at ELO SAIL that could affect their 
academic performance, such as initial ability. Two, the ELO SAIL participants in this study varied 
in the level of attendance. The coding of attendance data at ELO SAIL varied across the years 
included in our analyses, and, therefore, we did not consider the data reliable enough to use. 
However, prior evaluations of ELO SAIL identified benefits only for students who attended four 
weeks or at least 16 days. By including students with both high and low attendance, the following 
results may under-estimate the impact of ELO SAIL. Three, for the measure of reading in the fall 
used for students in Grades 1 and 2, we calculated whether a student had an increase in book level 
or stayed the same compared to the prior spring. However, because there are not equal intervals 
between the book levels, the effort for a student to increase from one book level to the next may 
not be the same for every book level. 

Lastly, it should be noted that causality may not be inferred from this study due to the lack of an 
experimental design. The underlying assumption of an experimental design is to administer an 
intervention program (e.g., ELO SAIL) to only one group and use another group who did not 
receive the same type of treatment as a comparison. Although the comparison group in this study 
did not attend ELO SAIL, it is not known whether these students received a similar type of 
academic or enrichment program during the summer. 
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Results 

Findings for Question 1: What were the demographic characteristics of students in Grades 
K-2 who attended ELO SAIL in 2012, 2013, 2014, and 2015? 


Table 1 presents the demographic characteristics of students in Grades K-2 who attended ELO 
SAIL in four summers: 2012, 2013, 2014, and 2015. Across the four school years, more than 
14,000 students in Grades K-2 attended the ELO SAIL program. Across the four cohorts, about 
two thirds of the students were Hispanic/Latino (63%) and one quarter (25%) were Black or 
African American. About one half of them (51%) received ESOL, three quarters (77%) received 
FARMS, and one tenth (10%) received special education services. The percentage for each 
subgroup was similar in each year. 


Table 1 


Characteristics of ELO SAIL Students in Grades K-2 by School Year and Subgroup 



2012- 

2013 

2013- 

2014 

2014-2015 

2015- 

2016 

All cohorts 

N 

% 

N 

% 

N 

% 

N 

% 

N 

% 

Total 

2,871 

100.0 

3,545 

100.0 

4,072 

100.0 

3,836 

100.0 

14,324 

100.0 

Grade level 


Kindergarten 

1,047 

36.5 

1,214 

34.2 

1,469 

36.1 

1,341 

35.0 

5,071 

35.4 

Grade 1 

993 

34.6 

1,232 

34.8 

1,354 

33.3 

1,327 

34.6 

4,906 

34.3 

Grade 2 

831 

28.9 

1,099 

31.0 

1,249 

30.7 

1,168 

30.4 

4,347 

30.3 

Gender 


Female 

1,348 

47.0 

1,651 

46.6 

1,910 

46.9 

1,835 

47.8 

6,744 

47.1 

Male 

1,523 

53.0 

1,894 

53.4 

2,162 

53.1 

2,001 

52.2 

7,580 

52.9 

Race/ethnicity 


American Indian 

— 

— 

13 

0.4 

12 

0.3 

13 

0.3 

40 

0.3 

Asian 

207 

7.2 

236 

6.7 

248 

6.1 

226 

5.9 

917 

6.4 

Black or African American 

658 

22.9 

843 

23.8 

1,042 

25.6 

1,039 

27.1 

3,582 

25.0 

Hispanic/Latino 

1,865 

65.0 

2,278 

64.3 

2,580 

63.4 

2,350 

61.3 

9,073 

63.3 

White 

92 

3.2 

124 

3.5 

121 

3.0 

133 

3.5 

470 

3.3 

Two or More Races 

44 

1.5 

50 

1.4 

68 

1.7 

75 

2.0 

237 

1.7 

Receipt of services during school year 


ESOL 

1,553 

54.1 

1,898 

53.5 

2,093 

51.4 

1,796 

46.8 

7,340 

51.2 

FARMS 

2,504 

87.2 

3,080 

86.9 

3,526 

86.6 

3,077 

80.2 

12,187 

85.1 

Special education 

275 

9.6 

335 

9.4 

408 

10.0 

360 

9.4 

1,378 

9.6 


Note. Numbers are not shown (— ) for groups <10. 
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Findings for Question 2: How did the students who participated in ELO SAIL in 2012, 2013, 
2014, and 2015 perform in the fall, compared with students in Title I schools who did not 
participate? 

Findings are separated by grade level; each grade level includes findings on reading performance, 
mathematics performance, and variations among student subgroups. This section includes graphs 
only for the total group; graphs for seven subgroups (Asian, Black or African American, 
Hispanic/Latino, White, ESOL recipients, FARMS recipients, and special education recipients) 
are in appendices for each academic area in each grade level. 

Kindergarten 


Reading. For the years of this study, relatively few kindergarten students (about 15%) had 
a book level from the AP-PR in fall. Therefore, instead of analyzing each student’s book level, 
the analyses focused on whether students were reading or not. Among kindergarten students, 
slightly more ELO SAIL participants than nonparticipants were reading in the fall of each year 
(Figure 2.0). None of these differences were statistically significant. 


Figure 2.0 

Percentage of kindergarten students who were reading, based on AP-PR in fall, 
for ELO SAIL participants and nonparticipants by school year 
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As seen in Appendix A, there was little variation in reading performance of ELO SAIL participants 
across subgroups of kindergarteners. As with all kindergarteners, there were more readers in ELO 
SAIL every year for four subgroups: Black or African American, Hispanic/Latino, ESOL 
recipients, and FARMS recipients. Better performance by ELO SAIL attendees also was apparent 
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for Asian students in three years and for special education recipients in two years. The following 
differences in favor of ELO SAIL attendees were statistically significant: 

• Asian students in 2015-2016 (% 2 (1) = 6.40, p < .05) 

• Black or African American students in 2013-2014 (% 2 (1) = 5.69, p < .05) 

• Hispanic/Latino students in 2013-2014 (% 2 (1) = 7.34, p < .01) 

• ESOL recipients in 2013-2014 (x 2 ( 1) = 4.53, p < .05) 

• FARMS recipients in three years: 2013-2014 (x 2 (l) = 5.09, p < .05), 2014-2015 (x 2 (l) = 
4.64, p < .05), and 2015-2016 (x 2 (l) = 5.39, p < .05) 


For White students, the pattern was reversed; in every year there were fewer readers among ELO 
SAIL attendees than non-attendees, although the differences were not statistically significant. 

More information on reading performance was available for AP-PR in 2015-2016, so that students 
without a book level were categorized as pre-reading or not reading. A higher percentage of 
kindergarteners who attended ELO SAIL were at a pre-reading level, compared to their peers who 
did not attend (Figure 2.1). This difference was statistically significant (x 2 (2) = 14.85, p < .001). 


Figure 2. 1 

Percentage of kindergarten students in reading categories from fall AP-PR for 2015-2016 

by ELO SAIL participation 



Reading category in fall 2015-2016*** 


Note. N for SAIL = 1,288. N for non-SAIL = 1,21 1. 
*p < .05, **p < .01, ***p < . 001 


Analyses of the three reading categories from fall 2015-2016 for kindergarten subgroups revealed 
little variation (Table A2). In five of the seven subgroups, there were more pre-readers among 
ELO SAIL attendees than non-attendees. Each difference was statistically significant, as follows: 

• Black or African American students in 2015-2016 (x 2 (2) = 14.96, p < .001) 
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• Hispanic/Latino students in 2015-2016 (% 2 (2) = 1 1.36, p < .01) 

• ESOL recipients in 2015-2016 (% 2 (2) = 23.37,p<.001) 

• FARMS recipients in 2015-2016 (% 2 (2) = 26.98, p < .001) 

• Special education recipients in 2015-2016 (% 2 (2) = 9.73, p < .01) 

Mathematics. Figure 2.2 presents the mean RIT score in mathematics from fall MAP-P 
testing for kindergarten students who participated in ELO SAIL and for students who did not. 
(Figure 2.2 excludes results from 2015-2016 because there were too few scores. See Table D1 in 
Appendix.) For each of the three years, the mean RIT score was lower for SAIL participants than 
for nonparticipants. This difference was statistically significant for two years in favor of non-ELO 
SAIL students: 2013-2014 (t (2,735) = -2.78. p < .01) and 2014-2015 (t (2,620) = 
-2.20. p < .05). 


Figure 2.2 

Mean MAP-P RIT scores in mathematics from fall for kindergarten students 
by participation in ELO SAIL and school year 
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Further analyses showed differential patterns of achievement in mathematics among most of the 
kindergarten subgroups (Appendix J). The findings for Asian and White students were the same 
as those for all students: ELO SAIL students had lower mean scores each year. The difference for 
White students in 2013-2014 was statistically significant in favor of non-attendees (t(216) = -2.54, 
p < .05). By contrast, ELO SAIL participants in four subgroups had higher mean mathematics 
scores than nonparticipants every year: Black or African American, Hispanic/Latino, ESOL 
recipients, and FARMS recipients. The difference for Black or African American students in 
2012-2013 was statistically significant in favor of ELO SAIL (t(570) = 2.43, p < .05). The 
differences were not statistically significant for the other three subgroups. Among special 
education recipients, SAIL participants had higher mean scores than nonparticipants for the first 
two years but had lower mean scores in 2014-2015. This latter difference was statistically 
significant in favor of non-attendees (t(208) = -2.94 p < .01). 
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Grade 1 

Reading. To capture changes in reading performance, analyses focused on whether each 
student had a fall book level that was higher or the same as his or her book level in spring; in other 
words, whether the student avoided any summer loss in reading. Figure 2.3 displays the percentage 
of Grade 1 students without summer loss in reading for ELO SAIL participants and 
nonparticipants. Specifically, the results for 2012-2013 indicated that 77% of the 691 students 
who attended ELO SAIL had a book level in fall 2012 (after ELO SAIL) that was higher or the 
same as their book level in spring 2012 (before ELO SAIL), while 75% of the 1,162 
nonparticipants had a fall book level that was higher or the same as their book level in spring. 
Thus, more ELO SAIL participants than nonparticipants avoided summer loss in 2012-2013. 
Further, this pattern of better performance by ELO SAIL attendees was the same for 2013-2014 
and 2015-2016. However, the difference between the two groups in favor of ELO SAIL was 
statistically significant in only one year 2015-2016 (% 2 (1) = 4.46, p < .05). By contrast, in 2014- 
2015, more nonparticipants than ELO SAIL participants had no summer loss, although this 
difference was not statistically significant. 


Figure 2.3 

Percentage of Grade 1 students whose AP-PR book level book level in fall increased or stayed the same 
from prior spring for ELO SAIL participants and nonparticipants by school year 


100 



2012-2013 2013-2014 2014-2015 2015-2016* 

School year 


Note. (#) refers to total number of students in group. 
*p < .05, **p < 01, ***p <. 001 


There was little variation in the relative performance of ELO SAIL participants among subgroups 
of 1 st graders (Appendix C). More ELO SAIL students than non-SAIL students avoided summer 
reading loss in each of three years among five of the seven Grade 1 subgroups: Black or African 
American, Hispanic/Latino, ESOL recipients, FARMS recipients, and special education recipients. 
For the Asian and White subgroups, more ELO SAIL participants than nonparticipants avoided 
summer loss for two years. The difference for FARMS recipients in 2015-2016 was significant 
(% 2 (1) = 6.85, p < .01) in favor of participants. For all subgroups, ELO SAIL students had weaker 
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performance in 2014-2015, as measured by summer loss in reading. One difference in that year 
was statistically significant: for special education recipients (% 2 (1) = 4.28, p < .05) in favor of 
nonparticipants. No other differences for subgroups of Grade 1 students were statistically 
significant. 

Mathematics. For mathematics, the analyses for first graders examined changes in RIT 
scores between spring (prior to ELO SAIL) and fall (after ELO SAIL). These changes were 
summarized by calculating the mean gain across each group of students, as seen in Figure 2.4. 
Specifically, the results for 2012-2013 indicated that, on average, students who attended ELO 
SAIL increased their RIT score by 5.2 points in fall 2012 (after ELO SAIL) compared to their 
score in spring 2012 (before ELO SAIL), while nonparticipants, on average, increased their RIT 
score by 4.4 points. Thus, the mean RIT gain was higher for ELO SAIL participants than for 
nonparticipants in 2012-2013. This pattern of better performance by ELO SAIL attendees was 
the same for the other years examined and was statistically significant in favor of ELO SAIL in 
both 2014-2015 (t(2,302) = 4.35, p < .001) and 2015-2016 (t(2,307) = 4.09, p < .001). 

Figure 2.4 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 ELO SAIL participants and nonparticipants by school year 
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There was little variation in the stronger performance of ELO SAIL participants when analyzing 
subgroups of 1 st graders (Appendix D). As with all Grade 1 students, mean gains in mathematics 
scores were higher among ELO SAIL students than non-ELO SAIL students in every year for 
every subgroup, except for Asian students in 2012-2013. Many of these differences were 
statistically significant in favor of ELO SAIL: 

• Asian students in one year: 2015-2016 (t(l 17) = 2.11 ,p< .05) 

• Black or African American students in one year: 2014-2015 (t(561) = 2.13, p < .05) 
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• Hispanic/Latino students in two years: 2014-2015 (t(l,348) = 3.69, p < .001) and 
2015-2016 0(1,461) =3.71, p < .001). 

• ESOL recipients in three years: 2013-2014 (t(l,193) = 2.02 , p < .05), 2014-2015 0(1,204) 
= 4.05, p < .001), and 2015-2016 0(1,204) = 3.03, p < .01) 

• FARMS recipients in two years: 2014-2015 (t(l,770) = 4.53, p < .001) and 2015-2016 
(t(l, 821) =3.91, p<. 001) 

• Special education recipients in two years: 2014-2015 (t(192) = 2.02, p < .05) and 
2015-2016 (t(180) =2.16, p<. 05). 

Grade 2 

For Grade 2, the analyses concerned changes in performance in the fall after each ELO session, 
compared to performance in the spring prior to that ELO session, as was done with Grade 1. 

Reading. Figure 2.5 displays the percentage of Grade 2 students without summer loss in 
reading by participation in ELO SAIL. Specifically, the results for 2012-2013 indicated that 65% 
of 2 nd graders who attended ELO SAIL had a book level in fall 2012 (after ELO SAIL) that was 
higher or the same as their book level in spring 2012 (before ELO SAIL), while 61% of 
nonparticipants had a fall book level that was higher or the same as their book level in spring. 
Thus, more ELO SAIL participants had no summer loss in 2012-2013 compared to 
nonparticipants. This pattern of higher performance by ELO SAIL attendees was the same for 
2013-2014 and 2015-2016, but in 2014-2015, slightly more nonparticipants than ELO SAIL 
participants had no summer loss. However, none of the differences was statistically significant. 


Figure 2.5 

Percentage of Grade 2 students whose AP-PR book level in fall increased or stayed the same 
from prior spring for ELO SAIL participants and nonparticipants by school year 
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Analyses of summer loss in reading for subgroups of Grade 2 by participation in ELO identified 
few variations (Appendix E). More attendees than non-attendees avoided summer reading loss in 
each year among the following four subgroups: Black or African American, Hispanic/Latino, 
ESOL recipients, and FARMS recipients. For the remaining three subgroups (i.e., Asian, White, 
and special education recipients), 2 nd graders who attended ELO SAIL had less summer loss 
compared to non-attendees in only two years. The following three differences were statistically 
significant in favor of ELO SAIL: 

• Black or African American students in 2012-2013 (x 2 (l) = 5.13, p < .05) 

• ESOL recipients in 2013-2014 (% 2 (1) = 4.33, p < .05) 

• FARMS recipients in 2015-2016 (% 2 (1) = 6.40, p < .05). 

Mathematics. For 2 nd graders, analyses for mathematics examined changes in RIT scores 
between spring (prior to ELO SAIL) and fall (after ELO SAIL), as was done for first graders. As 
seen in Figure 2.6, the changes were summarized as the mean gain for each group of students. 
Specifically, the results for 2012-2013 indicated that, on average, students who attended ELO 
SAIL increased their RIT score by 3.4 points in fall 2012 (after ELO SAIL) compared to their 
score in spring 2012 (before ELO SAIL), while nonparticipants, on average, increased their RIT 
score by 2.9 points. Thus, the mean gain in math scores was higher for ELO SAIL participants 
than for nonparticipants. This pattern of better performance by ELO SAIL attendees was the same 
for the other years examined. Further, the difference between the two groups was statistically 
significant in three years in favor of attendees: 2013-2014 (t(2,150) = 2.80, p < .01), 2014-2015 
(t(2,3 19) = 2.38, p < .05), and 2015-2016 (t(2,330) = 3.90, p < .001). 

Figure 2.6 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 ELO SAIL participants and nonparticipants by school year 
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There was little variation in the relative performance of ELO SAIL participants when analyzing 
subgroups of second graders (Appendix F). As with all Grade 2 students, SAIL attendees had 
higher gains than non-attendees every year for four of the seven subgroups: Black or African 
American, Hispanic/Latino, ESOL recipients, and FARMS recipients. Most of these differences 
were statistically significant in favor of ELO SAIL: 

• Black or African American students in 2015-2016 (t(610) = 2.66, p < .01) 

• Hispanic/Latino students in two years: 2014-2015 (t(1450) = 2.56, p < .05) and 2015-2016 
(t(l, 355) = 3.01, pc. 01) 

• ESOL recipients in three years: 2013-2014 (t(l,123) = 2.14 , p < .05), 2014-2015 (t( 1,166) 
=2.09, p < .05), and 2015-2016 (t(l,158) = 3.18, p < .001) 

• FARMS recipients in three years: 2013-2014 (t( 1,690) = 2.99, p < .01), 2014-2015 
(t(l ,785) =1.97, p < .05) , and 2015-2016 (t(l,836) = 3.84, p < .001). 

Among special education recipients, ELO SAIL participants had a greater mean gain in 
mathematics score during three school years. Among White students, ELO SAIL participants had 
higher performance only during the first two years analyzed; the difference for 2013-2014 was 
statistically significant in favor of ELO SAIL (t(148) = 2.97, p < .05). There was no consistent 
pattern of differences for Asian students. 

Summary for Question 2 

Kindergarten. Among all kindergarten students, the only significant difference in fall AP- 
PR reading results was in one of the four years, when more ELO SAIL attendees were at a pre- 
reading level (i.e., have mastered foundational reading skills) compared to their peers who did not 
attend. The impact on fall reading results was positive for kindergarten subgroups. In six of the 
seven subgroups (all except for White students), there were statistically significant differences for 
at least one year in favor of stronger reading performance by ELO SAIL attendees. 

Among all kindergarteners, the mean RIT score in mathematics from the fall MAP-P test was 
significantly lower for ELO SAIL participants than for nonparticipants in two of the three years 
examined. However, among subgroups of kindergarteners, the math performance in the fall of 
students who attended ELO SAIL did not differ from non-attendees, with very few exceptions. 

Grades 1 and 2. The analyses for Grade 1 and Grade 2 students concerned changes in 
performance in the fall after each ELO SAIL session, compared to performance in the spring prior 
to that ELO SAIL session; four years were examined. The reading measures concerned whether 
students increased or stayed at the same AP-PR book level. Among all first graders, there was a 
significant difference in reading for only one year; ELO SAIL participants experienced less 
summer loss in reading compared with their peers who did not attend. Among all second graders, 
there were no differences between ELO SAIL attendees and non-attendees in summer reading loss 
for any year. There was little variation in summer reading loss by subgroups of first or second 
graders; ELO SAIL attendees did not differ significantly from non-attendees, with very few 
exceptions. 
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The mathematics measures for Grades 1 and 2 concerned gains in RIT scores in mathematics from 
MAP-P. For all Grade 1 students, the mean gains of ELO SAIL attendees were significantly higher 
than those of non-attendees in two of the four years. The positive impact of ELO SAIL for two 
years also was evident in subgroups of first graders. Differences in math gains were statistically 
significant in favor of ELO SAIL for at least two years among five of the seven subgroups: Black 
or African American, Hispanic/Latino, ESOL recipients, FARMS recipients, and special education 
recipients. 

Among all second graders, the mean gains in math scores were significantly higher for ELO SAIL 
participants than for nonparticipants in three years. There was some variation among subgroups. 
Differences between attendees and non-attendees in Grade 2 were statistically significant in favor 
of ELO SAIL in three years for two subgroups, ESOL recipients and FARMS recipients, but in 
only one or two years for three subgroups: Black or African American, Hispanic/Latino, and White 
students. 
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Findings for Question 3: How did the Title I students who participated in ELO SAIL in 2012, 
2013, and 2014 perform in reading and mathematics at the end of the school year, compared 
with their nonparticipating peers in Title I schools? 

Findings are presented first for reading and then mathematics, with figures only for the total group. 
Figures for seven subgroups (Asian, Black or African American, Hispanic/Latino, White, ESOL 
recipients, FARMS recipients, and special education recipients) are in appendices. 

Reading 

The measure of reading performance was meeting the end-of-year grade-level reading benchmark 
on AP-PR (i.e., Level 4 for kindergarten, Level 16 for Grade 1, and Level M for Grade 2). 

Kindergarten. Among kindergarten students, a higher percentage of SAIL participants 
(87% in each year) met the end-of-year benchmark in reading than nonparticipants (85% in 2012- 
2013 and 82% in 2013-2014 and 2014-2015) in each of the three years (Figure 3.0). This 
difference was statistically significant in two years: 2013-2014 (% 2 (1) = 11.95, p < .001) and 
2014-2015 (x 2 (1) = 15.00, p < .001). 


Figure 3.0 

Percentage of kindergarten students who met the end-of-year AP-PR reading benchmark 
for ELO SAIL participants and nonparticipants by school year 



Note. (#) refers to total number of students in group. 
< .05, <. 01, ***p <. 001 


There was little variation in the reading performance of ELO SAIL participants versus 
nonparticipants for kindergarten subgroups (Appendix G). As with all kindergarteners, more ELO 
SAIL students than their nonparticipating peers met the end-of-year reading benchmark each year 
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among five subgroups: Black or African American, Hispanic/Latino, ESOL recipients, FARMS 
recipients, and special education recipients. Also, more Asian students who attended the summer 
program met the reading benchmark than non-attendees for two of the three years. Most of these 
differences in favor of ELO SAIL were statistically significant: 

• Black or African American students in all three years: 2012-2013 (% 2 (1) = 4.74, p < .05); 
2013-2014 (x 2 (l) = 5.05, p < .05); and 2014-2015 (x 2 (l) = 14.44, p < .001). 

• Hispanic/Latino students in two years: 2013-2014 (% 2 (1) = 1 1.63, p < .001) and 2014-2015 
(% 2 (1) = 8.92, p < .01). 

• ESOL recipients in two years: 2013-2014 (x 2 (l) = 13.29, p < .001) and 2014-2015 (x 2 (l) 
= 17.20, p < .001). 

• FARMS recipients in two years: 2013-2014 (x 2 (l) = 12.25, p < .001) and 2014-2015 (x 2 (l) 
= 21.95, p < .001). 

• Special education recipients in two years: 2013-2014 (x 2 (l) = 8.91, p < .01) and 2014- 
2015 (x 2 (l) = 12.50, p<. 001). 


The only variation involved White kindergarteners; among this subgroup, better performance in 
reading by ELO SAIL students occurred for only one year. 

Grade 1. Grade 1 ELO SAIL participants performed slightly lower than their 
nonparticipating counterparts in each year analyzed, as measured by meeting the end-of-year 
reading benchmark (Figure 3.1). None of these differences were statistically significant. 


Figure 3.1 

Percentage of Grade 1 students who met the end-of-year AP-PR reading benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Further analyses showed differential patterns of achievement among Grade 1 subgroups (Appendix 
H). Among five subgroups (Asian, Black or African American, plus students receiving ESOL, 
FARMS, or special education), more ELO participants than nonparticipants reached the end of the 
year reading benchmark in two years, usually 2013-2014 and 2015-2016. However, among White 
students, SAIL participants performed slightly lower than their nonparticipant counterparts in each 
year analyzed; there was no consistent pattern among Hispanic/Latino students. Across all these 
comparisons, two were significant, both in favor of ELO SAIL participants receiving special 
education services in 2012-2013 (% 2 (1) = 7.73, p < .01) and in 2013-2014 (% 2 (1) = 4.25, 
p < .05). 

Grade 2. Overall, Grade 2 ELO SAIL participants performed slightly lower in 2012-2013 
than their non- ELO SAIL peers in meeting the end-of-year reading benchmark, but outperformed 
nonparticipants in 2013-2014 and 2014-2015 (Figure 3.2). None of these differences were 
statistically significant. 


Figure 3.2 

Percentage of Grade 2 students who met the end-of-year AP-PR reading benchmark 
for ELO SAIL participants and nonparticipants by school year 
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The reading performance of ELO SAIL participants versus nonparticipants varied across Grade 2 
subgroups in 2012-2013 but showed little variation in the other two years studied (Appendix I). 
In 2012-2013, among four subgroups (i.e., Asian, Hispanic/Latino, ESOL recipients, FARMS 
recipients), more ELO SAIL participants than nonparticipants met the end-of-year reading 
benchmark, while in two other subgroups (i.e.. Black or African American, students receiving 
special education services), fewer participants met the benchmark. In both 2013-2014 and 2014- 
2015, more ELO SAIL attendees met the reading benchmark, compared to non-attendees, in five 
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or six subgroups. Out of all these comparisons, two were significant, both in favor of ELO SAIL 
participants in 2013-2014; one for students receiving ESOL services (% 2 (1) = 4.9P1 ,p < .05) and 
the second for students receiving FARMS services (% 2 (1) = 4.91, p < .05). 

Mathematics 


The measure to analyze mathematics performance at the end of the year was the MAP-P RIT score 
in mathematics from the spring following each ELO SAIL session. 

Kindergarten. On average, the end-of-year RIT scores in mathematics of ELO SAIL 
participants were lower than nonparticipants in each year analyzed (Figure 3.3). The difference in 
2013-2014 was statistically significant (t(2,931) = -2.54, p < .05). 


Figure 3.3 

Mean end-of-year MAP-P RIT scores in mathematics for kindergarten students 
by participation in ELO SAIL and school year 
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As seen in Appendix J, there were some variations by subgroups. There were almost no differences 
between ELO SAIL attendees and non-attendees in each year for Hispanic/Latino students, ESOL 
recipients, and special education recipients, except for 2014-2015. While both Black or African 
American students and FARMS recipients who attended ELO SAIL performed at least slightly 
better than their non-SAIL participant peers in every year, Asian and White SAIL students 
performed worse in each year. Three of these differences were statistically significant in 
2014-2015: 

• Black or African American students in favor of ELO SAIL: (t(670) = 2.73, p < .01) 

• White students in favor of non-ELO SAIL: (t(l 81) = -2.08, p < .05) 

• FARMS recipients in favor of ELO SAIL: (t(2,136) = 2.60, p < .01). 
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Grade 1. On average, the end-of-year RIT scores in mathematics of ELO SAIL first graders 
were lower than non-participating first graders in each year analyzed (Figure 3.4). The difference 
was statistically significant, in favor of non-SAIL students, in each year: 2012-2013 (t(2,535) = 
-3.01, p < .01), 2013-2014 (t(2,925) = -4.80, p < .001), and 2014-2015 (t(2,757) = -4.06, p < .01). 
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Mean end-of-year MAP-P RIT scores in mathematics for Grade 1 students 
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The lower performance by ELO SAIL participants compared to nonparticipants differed for some 
Grade 1 subgroups in 2012-2013, but not in the other two years (Appendix K). In 2012-2013, 
some subgroups of ELO SAIL students had higher, mean end-of-year scores in mathematics, than 
non-ELO students, while there was almost no difference for other subgroups, and both Black or 
African American and White participants had lower mean scores. In each year, the mean scores 
of White students who attended ELO SAIL were lower than non-attendees; these differences were 
statistically significant in favor of nonparticipants in two years: 2012-2013 (t(l 88) = -4.11, 
p < .001) and in 2013-2014 (t(186) = -2.58, p < .05). In both 2013-2014 and 2013-2014, there 
was little variation across subgroups; ELO SAIL participants in six of the seven subgroups had 
lower performance than nonparticipants. In 2013-2014, two differences (in addition to that for 
White students) were statistically significant in favor of non-ELO SAIL attendees: students 
receiving ESOL services (t(l ,8 15) = -2.01, p < .05) and students receiving FARMS (t(2,219) = 
-2.19, p < .05). 
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Grade 2. As with the findings for Grade 1, ELO SAIL participants in Grade 2 performed lower 
than their nonparticipant peers, as measured by mean RIT scores in mathematics, for each year 
analyzed (Figure 3.5). The difference was statistically significant, in favor of non-SAIL students, 
in each year: 2012-2013 (t(2,217) =-4.09, p < .001), 2013-2014 (t(2,808) = -2.76, p < .01), and 
2014-2015 (t(2,749) = -3.45, p < .001). 


Figure 3.5 

Mean end-of-year MAP-P RIT scores in mathematics for Grade 2 students 
by participation in ELO SAIL and school year 
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Appendix L presents figures with mean, end-of-year math scores for each subgroup of Grade 2 
students. As with all second graders, ELO SAIL participants in every subgroup had lower 
performance compared to nonparticipants in 2012-2013. For that year, the difference for students 
receiving special education services was statistically significant (t(213) = -2.25, p < .05), in favor 
of non-ELO SAIL participants. In the other two years, some ELO SAIL subgroups had better 
performance than their non-ELO SAIL peers, including Black or African American students; the 
difference in 2014-2015 for this subgroup was statistically significant, in favor of ELO SAIL 
(t(666) = 2.02, p < .05). By contrast, for two subgroups, Hispanic/Latino and White, ELO SAIL 
participants had lower mean end-of-year math scores than nonparticipants in both 2013-2014 and 
2014-2015, as well as 2012-2013. The difference for White students was statistically significant 
in 2014-2015 (t(164) = -3.63, p < .001), in favor of nonparticipants. 
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Summary for Question 3 

Reading. With respect to reading performance at the end of the year, the only statistically 
significant differences for all students involved kindergarteners; more kindergarteners who 
attended ELO SAIL met the end-of-the-year AP-PR reading benchmark than non-attendees, in two 
of the three years analyzed. First and second graders who participated in ELO SAIL performed as 
well in meeting the end-of-the-year reading benchmark as their peers in Title I schools who did 
not attend. 

The impact of ELO SAIL on end-of-year reading performance varied little across student 
subgroups. As among all kindergarteners, there were significant differences for two of the three 
years in favor of ELO SAIL among five of the seven subgroups: Black or African American, 
Hispanic/Latino, ESOL recipients, FARMS recipients, and recipients of special education 
services. For first graders and second graders, as with all students, attendees and non-attendees 
did not differ among most subgroups. However, there were a few variations (i.e., in favor of ELO 
SAIL attendees) that were statistically significant: in two years for first graders receiving special 
education services, in one year for second graders receiving ESOL services, and in one year for 
second graders receiving FARMS services. 

Mathematics. In analyses of all students, participants in ELO SAIL had lower scores on 
the MAP-P mathematics test at the end of the following year, on average, than their 
nonparticipating peers for each grade level in each of the three years examined. These differences 
in mean RIT scores in mathematics were statistically significant in favor of non-attendees in each 
year for Grades 1 and 2 but in only one year for kindergarten students. 

In contrast, among most subgroups, ELO SAIL attendees and non-attendees did not differ on end- 
of-the-year mathematics performance. However, there were a few statistically significant 
differences for one year in favor of non-attendees among subgroups of first and second graders as 
follows: first graders receiving ESOL services, first graders receiving FARMS, second graders 
receiving special education services, and White second graders. Also, among White first graders, 
there were significant differences in favor of non-attendees for two years. There were even fewer 
significant differences that were in favor of ELO SAIL attendees; each was for only one year and 
included three subgroups of kindergarteners (Black or African American, White, and FARMS 
recipients) and one subgroup of second graders (Black or African American). 
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Conclusion 

In conclusion, the positive impact of ELO SAIL was stronger on academic performance in the fall 
than on performance at the end of the year, based on statistically significant differences between 
attendees and non-attendees. Consistent with previous evaluations of ELO SAIL, the benefits 
varied by content area, with more positive findings in mathematics than in reading, and varied by 
grade level, such that positive findings in reading mainly were for kindergarteners, while positive 
findings in mathematics were limited mainly to first and second graders. However, in this study, 
unlike previous ones, a benefit for an entire grade level usually was evident for the majority of 
subgroups. Further, the following subgroups had more significant differences than other 
subgroups: African American or Black students, students receiving ESOL services, and students 
receiving FARMS services. 

Across four years of analysis, there was a positive impact on reading in the fall for kindergarten in 
one year, including all kindergarteners and the majority of subgroups, and for Grade 1 in one year, 
for all first graders, but not for subgroups. Because the positive impact for kindergarteners was in 
the only year with more detailed data on non-readers, the lack of positive findings in earlier years 
may reflect a lack of more detailed data. The only positive impact on end-of-the-year reading was 
for kindergarten in two of the three years studied; the stronger performance was apparent for all 
kindergarteners and the majority of subgroups. There were no significant differences in end-of- 
the year reading performance between ELO SAIL attendees and non-attendees in Grades 1 and 2. 

Out of four years analyzed, the positive impact of ELO SAIL on mathematics performance in the 
fall was evident for two or three years, for both Grades 1 and 2, among all students and the majority 
of subgroups in both grade levels. However, this benefit did not carry through to the spring; ELO 
SAIL participants had lower end-of-the-year mathematics scores than nonparticipants in three 
years for both first and second graders. These differences were evident for all students in each 
grade level, but not for most subgroups. For kindergarten students, there was no evidence for a 
positive impact of ELO SAIL on mathematics performance in the three years studied; attendees 
had lower scores than non-attendees in the fall for two years and at the end of the year for one 
year, although these differences were evident only for all kindergarteners, not for subgroups. 
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Appendix A 

Findings for Question 2 on Fall Performance: 
Reading for Subgroups of Kindergarten Students 

Table A1 

Number of Kindergarten Students With AP-PR Reading Data From Fall 


by School Year, Participation in ELO SAIL, and Selected Subgroups 


Kindergarten 

2012- 

■2013 

2013- 

-2014 

2014-2015 

2015- 

-2016 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

976 

1,713 

1,112 

1,860 

1,349 

1,478 

1,288 

1,211 

Race/ethnicity 




Asian 

59 

117 

74 

139 

64 

86 

76 

84 

Black or African American 

198 

428 

253 

487 

325 

358 

338 

313 

Hispanic/Latino 

666 

954 

717 

965 

897 

835 

51 

97 

White 

34 

164 

43 

187 

39 

150 

792 

671 

Receipt of services during school year 




ESOL 

615 

879 

700 

908 

845 

740 

770 

562 

FARMS 

813 

1,227 

917 

1,315 

1,124 

1,041 

1,027 

855 

Special education 

92 

136 

79 

170 

122 

121 

117 

99 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific Islander, 
and Two or More Races subgroups excluded due to small sample size. 


Figure A1 

Percentage of Asian American kindergarten students who were reading, based on AP-PR in fall, 
for ELO SAIL participants and nonparticipants by school year 
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Figure A2 

Percentage of Black or African American kindergarten students who were reading, based on AP-PR in 
fall, for ELO SAIL participants and nonparticipants by school year 
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Note. See Table A1 for number of students in each group. 
*p <. 05, **p < .01, ***/? < .001 


Figure A3 

Percentage of Hispanic/Latino kindergarten students who were reading, based on AP-PR in fall, 
for ELO SAIL participants and nonparticipants by school year 
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Figure A4 

Percentage of White kindergarten students who were reading, based on AP-PR in fall, 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table A1 for number of students in each group. 
*p <. 05, **p < .01, ***p < .001 


Figure A5 

Percentage of kindergarten students receiving ESOL services who were reading, based on AP-PR in fall, 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table A1 for number of students in each group. 
*p<.05, **p<.07, ***p<.001 
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Figure A6 

Percentage of kindergarten students receiving FARMS services who were reading, based on AP-PR in 
fall, or ELO SAIL participants and nonparticipants by school year 
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Note. See Table A1 for number of students in each group. 
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Figure A7 

Percentage of kindergarten students receiving special education services who were reading, 
based on AP-PR in fall, for ELO SAIL participants and nonparticipants by school year 


W) 

.G 

100 

90 


80 

<D 

C/3 

70 

3 

<D 

G 

60 

C/3 

G 

50 

0 ) 

H 

g 

40 

W) 

$h 

0 ) 

"O 

30 

G 

3 

20 

O 

10 


3.3 2.9 


2012-2013 



2013-2014 2014-2015 2015-2016 


School year 


SAIL 
Non-S AIL 


Note. See Table A1 for number of students in each group. 
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Table A2 

Categories of AP-PR Reading Performance for Kindergarten Students in 2015-2016 
by ELO SAIL Participation and Selected Subgroups 



Reading 

Pre-reading 

Not reading 


SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Kindergarten 

% 

% 

% 

% 

% 

% 

Total*** 

8.2 

7.2 

29.3 

23.1 

62.5 

69.7 

Race/ethnicity 



Asian 

17.1 

4.8 

35.5 

35.7 

59.5 

47.4 

Black or African American*** 

15.4 

12.1 

37.3 

25.6 

47.3 

62.3 

Hispanic/Latino* * * 

3.8 

3.6 

24.2 

17.1 

72.0 

79.3 

White 

13.7 

15.5 

31.4 

39.2 

54.9 

45.4 

Receipt of services during school year 



ESOL*** 

4.0 

2.8 

24.8 

14.6 

71.2 

82.6 

FARMS*** 

6.8 

4.3 

28.5 

20.0 

64.7 

75.7 

Special education*** 

3.4 

6.1 

26.5 

10.1 

70.1 

83.8 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, 
Pacific Islander, and Two or More Races subgroups excluded due to small sample size. 

<. 05, **p < .01, ***p < .001 
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Appendix B 

Findings for Question 2 on Fall Performance: 
Mathematics for Subgroups of Kindergarten Students 

Table B1 

Number of Kindergarten Students With MAP-P RIT Scores in Mathematics From Fall 


by School Year, Participation in ELO SAIL, and Se 

ected Subgroups 

2012-2013 

2013-2014 

2014-2015 

2015-2016 

Kindergarten SAIL Non 

SAIL Non 

SAIL Non 

SAIL Non 

Total 954 1,574 

1,083 1,654 

1,313 1,309 

11 5 

Race/ethnicity 




Asian 59 113 

Black or African American 189 383 

Hispanic/Latino 654 882 

White 34 150 

42 176 

248 405 

697 871 

42 176 

60 77 

320 319 

871 737 

38 132 

- 

Receipt of services during school year 




ESOL 602 807 

FARMS 794 1,121 

Special education 91 115 

679 811 
891 1,165 
76 140 

820 657 

1,094 923 

115 95 

- 


Note. Results not shown (— ) for a group with less than 5 students. 


Figure B1 

Mean MAP-P RIT scores in mathematics from fall for Asian kindergarten students 
by participation in ELO SAIL and school year 

155 
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School year 


Note. See Table B1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure B2 

Mean MAP-P RIT scores in mathematics from fall for Black or African American kindergarten students 

by participation in ELO SAIL and school year 
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Note. See Table B1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure B3 

Mean MAP-P RIT scores in mathematics from fall for Hispanic/Latino kindergarten students 
by participation in ELO SAIL and school year 
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Note. See Table B1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure B4 

Mean MAP-P RIT scores in mathematics from fall for White kindergarten students 
by participation in ELO SAIL and school year 
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Note. See Table B1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure B5 

Mean MAP-P RIT scores in mathematics from fall for kindergarten students receiving ESOL services 

by participation in ELO SAIL and school year 
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Figure B6 

Mean MAP-P RIT scores in mathematics from fall for kindergarten students receiving FARMS services 

by participation in ELO SAIL and school year 
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Note. See Table B1 for number of students in each group. 
*p<.05, **p<.07, ***p<.007 


Figure B7 

Mean MAP-P RIT scores in mathematics from fall for kindergarten students receiving 
special education services by participation in ELO SAIL and school year 



■ SAIL 
= Non-SAIL 


Note. See Table B 1 for number of students in each group. 
*p<.05, **p<.07, ***p<.007 
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Appendix C 

Findings for Question 2 on Fall Performance: 
Reading for Subgroups of Grade 1 Students 


Table Cl 

Number of Grade 1 Students With AP-PR Book Levels in Fall and Prior Spring 
by School Year, Participation in ELO SAIL, and Selected Subgroups 



2012- 

-2013 

2013- 

-2014 

2014-2015 

2015- 

-2016 

Grade 1 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

691 

1,162 

856 

875 

955 

949 

1,024 

909 

Race/ethnicity 




Asian 

55 

92 

56 

77 

75 

77 

53 

54 

Black or African American 

177 

280 

219 

194 

259 

242 

290 

213 

Hispanic/Latino 

418 

631 

529 

482 

562 

483 

635 

540 

White 

27 

122 

35 

91 

32 

110 

28 

71 

Receipt of services during school year 




ESOL 

388 

520 

463 

377 

523 

372 

549 

372 

FARMS 

570 

829 

676 

604 

767 

655 

840 

658 

Special education 

45 

59 

56 

38 

66 

40 

89 

32 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific Islander, and 
Two or More Races subgroups excluded due to small sample size. 


Figure Cl 

Percentage of Grade 1 Asian students whose AP-PR book level in fall increased or stayed the same 
from prior spring for ELO SAIL participants and nonparticipants by school year 
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Note. See Table Cl for number of students in each group. 
*p<.05, **p<.01, ***/?<. 001 
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3 

o 


Figure C2 

Percentage of Grade 1 Black or African American students 
whose AP-PR book level in fall increased or stayed the same from prior spring 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table Cl for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure C3 

Percentage of Grade 1 Hispanic/Latino students whose AP-PR book level in fall increased or 
sta yed the same from prior spring for ELO SAIL participants and nonparticipants by school y ear 
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Note. See Table Cl for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure C4 

Percentage of White Grade 1 students whose AP-PR book level in fall increased or stayed the same 
from prior spring for ELO SAIL participants and nonparticipants by school year 
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Note. See Table Cl for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure C5 

Percentage of Grade 1 ESOL recipients whose AP-PR book level in fall increased or stayed the 
same from prior spring for ELO SAIL participants and nonparticipants by school year 



School year 


Note. See Table Cl for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure C6 

Percentage of Grade 1 FARMS recipients whose AP-PR book level in fall increased or stayed the 
s ame from prior spring for ELO SAIL participants and nonparticipants by school year 



School year 


Note. See Table Cl for number of students in each group. 
*p<.05, **p<.01, ***p<. 001 


Figure C7 

Percentage of Grade 1 recipients of special education whose AP-PR book level in fall increased or 
sta yed the same from prior spring for ELO SAIL participants and nonparticipants by school y ear 
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Note. See Table Cl for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Appendix D 

Findings for Question 2 on Fall Performance: 
Mathematics for Subgroups of Grade 1 Students 


Table D1 

Number of Grade 1 Students With MAP-P RIT Scores in Mathematics From Fall and Prior Spring 
by School Year, Participation in ELO SAIL, and Selected Subgroups 



2012- 

2013 

2013- 

-2014 

2014-2015 

2015- 

-2016 

Grade 1 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

494 

720 

1,095 

1,117 

1,164 

1,140 

1,212 

1,097 

Race/ethnicity 




Asian 

30 

54 

64 

83 

78 

82 

58 

61 

Black or African American 66 

121 

250 

224 

288 

275 

317 

244 

Hispanic/Latino 

370 

469 

726 

673 

705 

645 

788 

675 

White 

21 

57 

36 

102 

38 

121 

30 

83 

Receipt of services 

during school year 





ESOL 

330 

367 

657 

538 

676 

530 

702 

504 

FARMS 

428 

556 

898 

806 

932 

840 

1,004 

819 

Special education 

47 

63 

107 

85 

112 

82 

117 

65 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific 
Islander, and Two or More Races subgroups excluded due to small sample size. 


Figure D1 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 Asian students by participation in ELO SAIL and school year 



Note. See Table D1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure D2 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 Black or African American students by participation in ELO SAIL and school year 



Note. See Table D1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure D3 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 Hispanic/Latino students by participation in ELO SAIL and school year 



Note. See Table D1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure D4 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 White students by participation in ELO SAIL and school year 
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Note. See Table D1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure D5 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 ESOL recipients by participation in ELO SAIL and school year 



■ SAIL 
= Non-SAIL 
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Note. See Table D1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure D6 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 FARMS recipients by participation in ELO SAIL and school year 
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Note. See Table D1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure D7 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 1 special education recipients by participation in ELO SAIL and school year 
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Note. See Table D1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Appendix E 

Findings for Question 2 on Fall Performance: 
Reading for Subgroups of Grade 2 Students 


Table El 

Number of Grade 2 Students With AP-PR Book Levels in Fall and Prior Spring 
by School Year, Participation in ELO SAIL, and Selected Subgroups 



2012- 

-2013 

2013- 

-2014 

2014-2015 

2015- 

-2016 

Grade 2 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

683 

1,215 

883 

1,077 

1,037 

1,241 

1,045 

1,213 

Race/ethnicity 




Asian 

58 

107 

67 

78 

57 

81 

69 

78 

Black or African American 160 

297 

219 

237 

257 

273 

292 

303 

Hispanic/Latino 

439 

658 

563 

605 

691 

729 

619 

701 

White 

16 

116 

23 

118 

21 

109 

35 

90 

Receipt of services 

during school year 





ESOL 

416 

572 

513 

480 

597 

533 

583 

537 

FARMS 

574 

857 

741 

783 

869 

881 

864 

923 

Special education 

61 

94 

95 

80 

105 

75 

97 

67 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific 
Islander, and Two or More Races subgroups excluded due to small sample size. 


Figure El 

Percentage of Grade 2 Asian students whose AP-PR book level in fall increased or stayed the same from 
prior spring for ELO SAIL participants and nonparticipants by school year 



Note. See Table El for number of students in each group. 
*p<.05, **p<.01, ***/?<. 001 
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Figure E2 

Percentage of Grade 2 Black or African American students whose AP-PR book level in fall increased or 
s tayed the same from prior spring for ELO SAIL participants and nonparticipants by school yea r 
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Note. See Table El for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure E3 

Percentage of Grade 2 Hispanic/Latino students whose AP-PR book level in fall increased or stayed the 
same from prior spring for ELO SAIL participants and nonparticipants by school year 
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Note. See Table El for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure E4 

Percentage of Grade 2 White students whose AP-PR book level in fall increased or stayed the same from 
prior spring for ELO SAIL participants and nonparticipants by school year 



Note . See Table El for number of students in each group. 
*p<.05, **p<.01, ***/?<. 001 


Figure E5 

Percentage of Grade 2 students receiving ESOL services whose AP-PR book level in fall increased or 
s tayed the same from prior spring for ELO SAIL participants and nonparticipants by school yea r 



■ SAIL 
= Non-S AIL 


2012-2013 2013-2014* 2014-2015 2015-2016 
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Note. See Table El for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure E6 

Percentage of Grade 2 students receiving FARMS services 
whose AP-PR book level in fall increased or stayed the same from prior spring 
for ELO SAIL participants and nonparticipants by school year 


3 

o 


<D 

3 




100 

90 

80 

70 

60 

50 

40 

30 

20 

10 



61.3 60.2 



65.9 65.6 




■ SAIL 
= Non-S AIL 


2012-2013 2013-2014 2014-2015 2015-2016* 


School year 


Note. See Table El for number of students in each group. 
*p<.05, **p<.07, ***p<.007 


Figure E7 

Percentage of Grade 2 students receiving special education services 
whose AP-PR book level in fall increased or stayed the same from prior spring 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table El for number of students in each group. 
*p<.d5, **p<.01, ***p<.001 
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Appendix F 

Findings for Question 2 on Fall Performance: 

Mathematics for Subgroups of Grade 2 Students 

Table FI 

Number of Grade 2 Students With MAP-P RIT Scores in Mathematics From Fall and Prior Spring 


by School Year, Participation in 

ELO SAIL, and Selected Subgrou] 

3S 


2012- 

2013 

2013- 

-2014 

2014-2015 

2015- 

-2016 

Grade 2 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

408 

642 

977 

1,175 

1,059 

1,262 

1,092 

1,240 

Race/ethnicity 




Asian 

32 

60 

72 

81 

58 

83 

74 

83 

Black or African American 75 

122 

227 

246 

258 

275 

303 

309 

Hispanic/Latino 

286 

387 

645 

676 

709 

743 

647 

710 

White 

7 

58 

21 

129 

23 

110 

36 

96 

Receipt of services 

during school year 





ESOL 

279 

341 

592 

533 

615 

553 

614 

546 

FARMS 

350 

468 

833 

859 

889 

898 

901 

937 

Special education 

48 

65 

116 

95 

121 

90 

105 

76 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific Islander, 
and Two or More Races subgroups excluded due to small sample size. 


Figure FI 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 Asian students by participation in ELO SAIL and school year 
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Note. See Table FI for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure F2 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 Black or African American students by participation in ELO SAIL and school y ear 
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Note. See Table FI for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure F3 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 Hispanic/Latino by participation in ELO SAIL and school year 
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Note. See Table FI for number of students in each group. 
*p<. 05, **p<.01, ***p<.001 
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Figure F4 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 White students by participation in ELO SAIL and school year 
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School year 


Note. See Table FI for number of students in each group. 
*p<. 05, **p<.01, ***p<.001 


Figure F5 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 ESOL recipients by participation in ELO SAIL and school year 



■ SAIL 
= Non-SAIL 
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2015-2016*** 


Note. See Table FI for number of students in each group. 
*p<. 05, **p<.01, ***p<.001 
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Figure F6 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 FARMS recipients by participation in ELO SAIL and school year 
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Note. See Table FI for number of students in each group. 
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Figure F7 

Mean gain in MAP-P RIT mathematics score from prior spring to fall for 
Grade 2 special education recipients by participation in ELO SAIL and school year 
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Appendix G 

Findings for Question 3 on End-of-Year Performance: 
Reading for Subgroups of Kindergarten Students 

Table G1 

Number of Kindergarten Students With End-of-Year AP-PR Reading Data by 


School Year, Participation in ELO SAI 

and Selected Subgroups 

2012-2013 

2013-2014 

2014-2015 

Kindergarten SAIL Non 

SAIL Non 

SAIL Non 

Total 976 1,713 

Race/ethnicity 

1,112 1,860 

1,349 1,478 

Asian 59 117 

Black or African American 198 428 

Hispanic/Latino 666 954 

White 34 164 

Receipt of services during school year 

74 139 

253 487 

717 965 

43 187 

64 86 

325 358 

897 835 

39 150 

ESOL 615 879 

FARMS 813 1,227 

Special education 92 136 

700 908 

917 1,315 

79 170 

845 740 

1,124 1,041 

122 121 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific 


Islander, and Two or More Races subgroups excluded due to small sample size. 


Figure G1 

Percentage of Asian kindergarten students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table G1 for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 
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Figure G2 

Percentage of Black or African American kindergarten students who met the end-of-year AP-PR reading 
benchmark for ELO SAIL participants and nonparticipants by school year 



2012-2013* 2013-2014* 2014-2015*** 


School year 


Note. See Table G1 for number of students in each group. 
*p<. 05, **p<.01, ***p<.001 


Figure G3 

Percentage of Hispanic/Latino kindergarten students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 



School year 


Note. See Table G1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure G4 

Percentage of White kindergarten students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table G1 for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 


Figure G5 

Percentage of kindergarten students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for ESOL recipients 
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Note. See Table G1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure G6 

Percentage of kindergarten students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for FARMS recipients 
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Note. See Table G1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure G7 

Percentage of kindergarten students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for special education recipients 
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Note. See Table G1 for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Program Evaluation 


55 


Impact of ELO SAIL: Fall 2012 to Fall 2015 


Montgomery County Public Schools 


Office of Shared Accountability 


Appendix H 

Findings for Question 3 on End-of-Year Performance: 
Reading for Subgroups of Grade 1 Students 


Table HI 

Number of Grade 1 Students With End-of-Year AP-PR Reading Data by 
School Year, Participation in ELO SAIL, and Selected Subgroups 



2012-2013 

2013- 

-2014 

2014-2015 

Grade 1 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

900 

1,685 

1,138 

1,827 

1,231 

1,540 


Race/ethnicity 


Asian 64 120 

Black or African American 201 411 

Hispanic/Latino 589 934 

White 31 164 

Receipt of services during school year 

64 140 

258 486 

759 994 

38 152 

89 100 

317 406 

752 824 

42 155 

ESOL 560 785 

FARMS 760 1,243 

Special education 92 149 

688 835 
938 1,309 
112 153 

719 691 

1,001 1,116 

123 113 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, 
Pacific Islander, and Two or More Races subgroups excluded due to small sample size. 


Figure HI 

Percentage of Asian Grade 1 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 



Note. See Table HI for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Figure H2 

Percentage of Black or African American Grade 1 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table HI for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 


Figure H3 

Percentage of Hispanic/Latino Grade 1 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table HI for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 
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Figure H4 

Percentage of White Grade 1 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 



Note. See Table HI for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 


Figure H5 

Percentage of Grade 1 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for ESOL recipients 
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Note. See Table HI for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 
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Figure H6 

Percentage of Grade 1 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for FARMS recipients 
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Figure H7 

Percentage of Grade 1 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for special education recipients 



Note. See Table HI for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Appendix I 

Findings for Question 3 on End-of-Year Performance: 
Reading for Subgroups of Grade 2 Students 


Table II 

Number of Grade 2 Students With End-of-Year AP-PR Reading Data by 
School Year, Participation in ELO SAIL, and Selected Subgroups 


Grade 2 

2012- 

-2013 

2013- 

-2014 

2014-2015 

SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

Race/ethnicity 

755 

1,504 

1,022 

1,821 

1,142 

1,626 

Asian 

63 

134 

75 

133 

62 

111 

Black or African American 172 

368 

238 

475 

282 

394 

Hispanic/Latino 

492 

816 

672 

969 

761 

919 

White 

17 

140 

24 

178 

24 

143 

Receipt of services 

during school year 






ESOL 

475 

715 

618 

793 

655 

721 

FARMS 

639 

1,065 

870 

1,315 

960 

1,164 

Special education 

74 

132 

125 

176 

136 

135 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific Islander, and 
Two or More Races subgroups excluded due to small sample size. 


Figure II 

Percentage of Asian Grade 2 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table II for number of students in each group. 
*p<. 05, **p<. 01, ***/?<. 001 


Program Evaluation 


60 


Impact of ELO SAIL: Fall 2012 to Fall 2015 


Montgomery County Public Schools 


Office of Shared Accountability 


Figure 12 

Percentage of Black or African American Grade 2 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table II for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 


Figure 13 

Percentage of Hispanic/Latino Grade 2 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table II for number of students in each group. 
*p<. 05, **p<.01, ***p<.001 
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Figure 14 

Percentage of White Grade 2 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year 
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Note. See Table II for number of students in each group. 
*p<. 05, **p<. 01, ***p<.001 


Figure 15 

Percentage of Grade 2 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for ESOL recipients 
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Note. See Table II for number of students in each group. 
*p<.05, **p<. 01, ***p<.001 
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Figure 16 

Percentage of Grade 2 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for FARMS recipients 
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Note. See Table II for number of students in each group. 
*p<.05, **p<.01, ***p<.001 


Figure 17 

Percentage of Grade 2 students who met the AP-PR end-of-year benchmark 
for ELO SAIL participants and nonparticipants by school year for special education recipients 
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Note. See Table II for number of students in each group. 
*p<.05, **p<.01, ***p<.001 
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Appendix J 

Findings for Question 3 on End-of-Year Performance: 
Mathematics for Subgroups of Kindergarten Students 


Table J1 

Number of Kindergarten Students With End-of-Year MAP-P RIT Score in Mathematics 
by School Year, Participation in ELO SAIL, and Selected Subgroups 




2012-2013 

2013- 

-2014 

2014-2015 

Kindergarten 


SAIL 

Non 

SAIL 

Non 

SAIL 

Non 

Total 

Race/ethnicity 


958 

1,675 

1,098 

1,835 

1,338 

1,445 

Asian 


59 

118 

72 

137 

62 

81 

Black or African American 

196 

414 

250 

471 

325 

347 

Hispanic/Latino 

White 


650 

34 

932 

161 

708 

43 

962 

185 

888 

39 

827 

144 

Receipt of services 
ESOL 

during school year 
671 

965 

773 

1,011 

942 

820 

FARMS 


796 

1,197 

905 

1,303 

1,115 

1,023 

Special education 


99 

149 

87 

173 

123 

105 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific 
Islander, and Two or More Races subgroups excluded due to small sample size. 


Figure J1 

Mean end-of-year MAP-P RIT scores in mathematics for Asian kindergarten students 
by participation in ELO SAIL and school year 
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Note. See Table J 1 for number of students in each group. 
*p<. 05, **£><.01, ***£><.001 
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Figure J2 

Mean end-of-year MAP-P RIT scores in mathematics for Black or African American 
kindergarten students by participation in ELO SAIL and school year 
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Note. See Table J 1 for number of students in each group. 
*p<. 05, **/?<.01, ***p<.001 


Figure J3 

Mean end-of-year MAP-P RIT scores in mathematics for Hispanic/Latino kindergarten students 
by participation in ELO SAIL and school year 
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Note. See Table J 1 for number of students in each group. 
*p<.05, **p<. 01, ***£><. 001 
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Figure J4 

Mean end-of-year MAP-P RIT scores in mathematics for White kindergarten students 
by participation in ELO SAIL and school year 
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Note. See Table J 1 for number of students in each group. 
*p<. 05, **£><.01, ***£><.001 


Figure J5 

Mean end-of-year MAP-P RIT scores in mathematics for kindergarten students 
who receive ESOL services by participation in ELO SAIL and school year 
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Note. See Table J 1 for number of students in each group. 
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Figure J6 

Mean end-of-year MAP-P RIT scores in mathematics for kindergarten students 
who receive FARMS services by participation in ELO SAIL and school year 
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Note. See Table J 1 for number of students in each group. 
*£><• 05, **£><. 01, ***p<.001 


Figure J7 

Mean end-of-year MAP-P RIT scores in mathematics for kindergarten students 
who receive special education services by participation in ELO SAIL and school year 
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Appendix K 

Findings for Question 3 on End-of-Year Performance: 
Mathematics for Subgroups of Grade 1 Students 

Table K1 

Number of Grade 1 Students With End-of-Year MAP-P RIT Score in Mathematics 


by School Year, Participation in ELO SAIL, and Selected Su 

Dgroups 

2012-2013 

2013-2014 

2014-2015 

Grade 1 SAIL Non 

SAIL Non 

SAIL Non 

Total 886 1,651 

Race/ethnicity 

1,127 1,800 

1,229 1,530 

Asian 62 119 

Black or African American 196 399 

Hispanic/Latino 583 918 

White 30 160 

Receipt of services during school year 

63 138 

255 477 

752 980 

38 150 

88 100 

317 402 

751 819 

41 156 

ESOL 659 934 

FARMS 749 1,225 

Special education 102 136 

809 1,008 

929 1,292 

127 166 

849 841 

1,001 1,108 
126 107 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific 
Islander, and Two or More Races subgroups excluded due to small sample size. 


Figure K1 

Mean end-of-year MAP-P RIT scores in mathematics for Asian Grade 1 students 
by participation in ELO SAIL and school year 
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Note. See Table K1 for number of students in each group. 
*p<. 05, **p<.01, ***p<. 001 
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Figure K2 

Mean end-of-year MAP-P RIT scores in mathematics for Black or African American Grade 1 students 

by participation in ELO SAIL and school year 
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Note. See Table K1 for number of students in each group. 
*£><. 05, **£><.01, ***£><.001 


Figure K3 

Mean end-of-year MAP-P RIT scores in mathematics for Hispanic/Latino Grade 1 students 
by participation in ELO SAIL and school year 
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Note. See Table K1 for number of students in each group. 
*p<. 05, **£><.01, ***£><.001 
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Figure K4 

Mean end-of-year MAP-P RIT scores in mathematics for White Grade 1 students 
by participation in ELO SAIL and school year 
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Note. See Table K1 for number of students in each group. 
*£><. 05, **£><.01, ***£><.001 


Figure K5 

Mean end-of-year MAP-P RIT scores in mathematics for Grade 1 students 
who receive ESOL services by participation in ELO SAIL and school year 
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Note. See Table K1 for number of students in each group. 
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Figure K6 

Mean end-of-year MAP-P RIT scores in mathematics for Grade 1 students 
who receive FARMS services by participation in ELO SAIL and school year 
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Note. See Table K1 for number of students in each group. 
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Figure K7 

Mean end-of-year MAP-P RIT scores in mathematics for Grade 1 students 
who receive special education services by participation in ELO SAIL and school year 
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Appendix L 

Findings for Question 3 on End-of-Year Performance: 
Mathematics for Subgroups of Grade 2 Students 

Table LI 

Number of Grade 2 Students With End-of-Year MAP-P RIT Score in Mathematics 


by School Year, Participation in ELO SAIL, and Selected Su 

Dgroups 

2012-2013 

2013-2014 

2014-2015 

Grade 2 SAIL Non 

SAIL Non 

SAIL Non 

Total 747 1,472 

Race/ethnicity 

1,019 1,791 

1,140 1,611 

Asian 63 132 

Black or African American 171 356 

Hispanic/Latino 486 803 

White 16 137 

Receipt of services during school year 

74 134 

238 462 

670 957 

24 173 

62 111 

282 386 

759 913 

24 142 

ESOL 547 824 

FARMS 631 1,045 

Special education 83 132 

749 1,007 

870 1,295 

140 174 

760 874 
959 1,156 
151 131 


Note. SAIL = participants in ELO SAIL. Non = nonparticipants in ELO SAIL. American Indian, Pacific 
Islander, and Two or More Races subgroups excluded due to small sample size. 


Figure LI 

Mean end-of-year MAP-P RIT scores in mathematics for Asian Grade 2 students 
by participation in ELO SAIL and school year 
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Figure L2 

Mean end-of-year MAP-P RIT scores in mathematics for Black or African American Grade 2 students 

by participation in ELO SAIL and school year 
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Note. See Table LI for number of students in each group. 
*£><• 05, **£><.01, ***£><.001 


Figure L3 

Mean end-of-year MAP-P RIT scores in mathematics for Hispanic/Latino Grade 2 students 
by participation in ELO SAIL and school year 
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Note. See Table LI for number of students in each group. 
*£><.05, **£><.01, ***£><.001 
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Figure L4 

Mean end-of-year MAP-P RIT scores in mathematics for White Grade 2 students 
by participation in ELO SAIL and school year 
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Note. See Table LI for number of students in each group. 
*p<. 05, **/?<.01, ***p<-001 


Figure L5 

Mean end-of-year MAP-P RIT scores in mathematics for Grade 2 students 
who receive ESOL services by participation in ELO SAIL and school year 
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Note. See Table LI for number of students in each group. 
*p<. 05, **p<.01, ***p<. 001 
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Figure L6 

Mean end-of-year MAP-P RIT scores in mathematics for Grade 2 students 
who receive FARMS services by participation in ELO SAIL and school year 
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Note. See Table LI for number of students in each group. 
*£><. 05, **£><.01, ***/?<. 001 


Figure L7 

Mean end-of-year MAP-P RIT scores in mathematics for Grade 2 students 
who receive special education services by participation in ELO SAIL and school year 
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Note. See Table LI for number of students in each group. 
*p<.05, **£><.01, ***£><.001 
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